Digital non-linear editing (DNLE) is a process by which digital media may be edited. DNLE, as the name implies, is performed on digital media stored as data in digital media files on a digital random access medium. DNLE may be conducted in a non-linear fashion because the digital media files in which the digital media is stored can be randomly accessed. Thus an editor may access a piece of the digital media without having to proceed sequentially through other pieces of the digital media stored in the same or other digital media files. More than one editor also may be able to access different pieces of the same digital media contemporaneously. The digital media may be a digitized version of a film or videotape or digital media produced through live capture onto a disk of a graphics or animation software application. Example commercial DNLE systems include the Media Composer(copyright) or Symphony video production systems or NewsCutter(copyright) news editing system available from Avid Technology, Inc. For a more detailed description of DNLE, see Digital Nonlinear Editing, New Approaches to Editing Film and Video, 1993 , by Thomas Ohanian.
Color modification is a class of operations that may be performed to correct color errors due to process errors and to adjust the colors used in the video for artistic expression. Such color modifications may include enhancing contrasts or color in an image to give a program an overall xe2x80x9clook,xe2x80x9d or applying special effects to selected segments. Other color modifications may be made by an editor during an editing session to correct problems with color or lighting resulting from the source of the media. Such corrections may include color balancing for camera and lighting differences, correcting for film processing differences, matching colors and tones from shot to shot, or adjusting video levels for differences in source tapes, source decks, etc.
Digital images are comprised of an array of picture elements called pixels. For a given image, color modifications may be applied to all pixels in the image or pixels comprising a portion of the image. In digital video signal processing, a variety of data formats can be used to represent the color of pixels within a digital image. Formats may be classified into two major categories: composite signals and component signals. Component formats represent a color as multiple components, each component defining a value along a dimension of the color space in which the color being represented is defined. A composite video signal is an analog signal that uses a high frequency subcarrier to encode color information. The subcarrier is a sinewave of which the amplitude is modulated by the saturation of the color represented by the signal, and the hue of the color is encoded as a phase difference from a color burst. Analog composite signals are generally used to broadcast television video signals.
There are a variety of component formats used to represent color. RGB (Red, Green, Blue) format represents a color with a red component, a green component and a blue component. CMYK (Cyan, Magenta, Yellow, Black) format represents a color with a cyan component, a magenta component, and a yellow component. CMYK is a format commonly used by printers. The CMYK components are color opposites of RGB components. In a three-dimensional coordinate system, each component of either the RGB or the CMY format represents a value along an axis, the combination of the values defining a cubic color space.
The data formats HSL (Hue, Saturation, Lightness or Luminance) and HSV (Hue, Saturation, Value) represent a color with a hue component, a saturation component, and a luma component. In a three-dimensional coordinate system, the luma component represents a value along a luma axis, the hue component represents the angle of a chroma vector with respect to the luma axis and the saturation component represents the magnitude of the chroma vector. The combination of the values defines a hexagonal cone-shaped color space.
YCrCb, YUV, and YIQ are three formats that represent a color with a luma component Y, and two chroma components, Cr and Cb, U and V, or I and Q, respectively, that define a chroma vector. In a three-dimensional coordinate system, each component of either the YCrCb, YUV, and YIQ format represents a value along an axis, the combination of the values defining a cylindrical color space around the luma axis. The chroma components define the chroma vector. In data formats with a luma component, the luma component can be used independently to represent a pixel in a black and white image to be displayed, for example, with a black and white monitor.
A typical color modification in HSL color space may include increasing a color component or a combination of color components for all pixels in each digital image of a section of digital media. Typically, an editor accesses a segment of a composition that represents the section of media through an editor interface and inputs desired color modifications through the editor interface. Some systems permit an editor to apply color modifications to only portions of a digital image. Portions of a digital image may be specified as one or more pixels or by specifying a region. For example, an editor may select with a mouse, keyboard, or some other editor input device a portion of the image and define color modifications for the selected portion. A suitable commercial system for color modification is Avid Media Illusion(trademark) available from Avid Technology, Inc. The Avid Media Illusion Reference Guide, available from Avid Technology, Inc. is herein incorporated by reference. Other commercial software applications may be used.
Some systems permit an editor to define a color modification as a function of the luma of a pixel. Some systems also allow a user to define functions of luma that allow an editor to define the effect of a color modification over a range of possible luma values of a pixel. For example, an editor may define a highlight function that primarily affects high luma values, a midtone function that primarily affects mid-range luma values, and a shadow function that primarily affects low luma values. The editor may then associate a color modification with each function. If more than one function is defined over a range of luma values, each function, and thus a color modification associated with the function, may have a weighted effect for a given luma value. This weighted effect is defined by the value of the function for the given luma value normalized with respect to the values of functions for the given luma value.
For example, a color modification A is specified for a luma function defined by X=2Lxe2x88x922, and a color modification B is specified for a luma function defined by Y=0.5L+2. For pixel P with a luma value of 16, X=30 and Y=10. Normalizing X and Y, the weighted value of X is 0.75 and the weighted value of Y is 0.25. Thus, the color modification applied to pixel P is Z=0.75A+0.25B. It should be noted that the equation used in this example is linear to simplify the explanation, but for decreasing and increasing the effect of a color modification for different values of luma, the function is usually non-linear.
The range of possible luma values may have ranges in which one of the functions has a predominant effect. For the highlight, midtone, and shadow functions described above, a highlight range is the range of luma values for which the highlight function has the strongest effect or the greatest weighted value, a midtone range is the range of luma values for which the midtone function has the strongest effect or the greatest weighted value, and the shadow range is the range of luma values for which the shadow function has the strongest effect or the greatest weighted value.
On some DNLE systems, an editor may specify a variety of color modifications to be applied to a digital image or a portion of a digital image. These modifications may be specified in variety of color formats, including RGB, HSL, and composite, and in a variety of units, depending on the interface used by the editor and in a variety of units. Possible units include IRE units in accordance with NTSC standards, millivolts in accordance with PAL standards, percentages, radians, and degrees, and a unit may be represented in integer or real number form. The units and formats for software, hardware, and storage on a color modification may all be different.
Types of color modifications may range from simple offsets and linear functions to complex non-linear functions and color effects defined by an editor, and may be applied to all pixels of a digital image or specified for pixels meeting positional or component-based criteria. For a component color, modifications may be specified for less than all of the components. For example, a color modification may specify that all pixels having a luma value within a certain range receive an increase in saturation depending on the luma value of the pixel. Color modifications may also include combining color components of a component color to produce a modified component or color. Color modifications are described in U.S. patent application Ser. No. 09/293,730, entitled xe2x80x9cSource Color Modification on a Digital Nonlinear Editing Systemxe2x80x9d (the Gonsalves I application) by Robert Gonsalves and Michael D. Laird filed on Apr. 16, 1999, and in U.S. patent application Ser. No. 09/293,732, entitled xe2x80x9cMulti-tone Representation of a Digital Image on a Digital Nonlinear Editing Systemxe2x80x9d (the Gonsalves II application), by Robert Gonsalves, filed on Apr. 16, 1999.
Because color modifications may be specified in a variety of ways and in a variety of combinations, performing color modifications on an image may include several computations. These computations may consume considerable resources, such as bandwidth and processor time, slowing down the color modification process, and consequently the processing of a video stream. Also, as the number of computations increases, the concatenation of the rounding errors inherent in these computations reduces the accuracy of the modifications applied to a pixel color. This reduction in accuracy may produce undesired artifacts in a digital image.
A color modification system that reduces the number of computations performed for color modifications increases the rate at which color modification may be performed and decreases the effects of rounding errors. Decreasing the effects of rounding errors produces more accurate color modifications, thereby reducing the likelihood of artifacts.
In one aspect, a system performs color modification on a color, where the color including a first, second, and third component. Each component of the color defines a value of the color. The system includes a chroma lookup table having a plurality of entries, each entry corresponding to a luma value and containing chroma coefficients. The chroma coefficients define color modifications to be applied to the components of the color. If a luma value is received, the chroma lookup table generates output chroma coefficients at an output. The chroma coefficients are generated by accessing the entry corresponding to the luma value, and extracting the output coefficients from the entry.
In one embodiment, the chroma coefficients include at least four matrix coefficients, and the system includes a first matrix multiplier that receives the four matrix coefficients and at least a first and second component of the color at an input. The matrix multiplier generates at least a first modified component and second modified component as output by applying matrix multiplication to the first and second components using the four output chroma coefficients as the coefficients of the matrices.
In another embodiment, the chroma lookup table defines a function of luma, where the function may be nonlinear.